Determining The Role Of Radiation Oncologist Demographic Factors On Segmentation Quality: Insights From A Crowd-Sourced Challenge Using Bayesian Estimation.

BACKGROUND: Medical image auto-segmentation is poised to revolutionize radiotherapy workflows. The quality of auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of these clinician-derived segmentations have yet to be fully understood or quantified. Therefore, the purpose of this study was to determine the role of common observer demographic variables on quantitative segmentation performance. METHODS: Organ at risk (OAR) and tumor volume segmentations provided by radiation oncologist observers from the Contouring Collaborative for Consensus in Radiation Oncology public dataset were utilized for this study. Segmentations were derived from five separate disease sites comprised of one patient case each: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and gastrointestinal (GI). Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus gold standard primarily using the Dice Similarity Coefficient (DSC); surface DSC was investigated as a secondary metric. Metrics were stratified into binary groups based on previously established structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Markov chain Monte Carlo Bayesian estimation were used to investigate the association between demographic variables and the binarized segmentation quality for each disease site separately. Variables with a highest density interval excluding zero - loosely analogous to frequentist significance - were considered to substantially impact the outcome measure. RESULTS: After filtering by practicing radiation oncologists, 574, 110, 452, 112, and 48 structure observations remained for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of observations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumor volumes, respectively. Bayesian regression analysis revealed tumor category had a substantial negative impact on binarized DSC for the breast (coefficient mean ± standard deviation: -0.97 ± 0.20), sarcoma (-1.04 ± 0.54), H&N (-1.00 ± 0.24), and GI (-2.95 ± 0.98) cases. There were no clear recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations and wide highest density intervals. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality. Future studies should investigate additional demographic variables, more patients and imaging modalities, and alternative metrics of segmentation acceptability.

Prognostic Value of the Intermediate-risk Feature in Men with Favorable Intermediate-risk Prostate Cancer: Implications for Active Surveillance.

Background: Guidelines suggest that active surveillance (AS) may be considered for select patients with favorable intermediate-risk (fIR) prostate cancer. Objective: To compare the outcomes between fIR prostate cancer patients included by Gleason score (GS) or prostate-specific antigen (PSA). Most patients are classified with fIR disease due to either a 3 + 4 = 7 GS (fIR-GS) or a PSA level of 10-20 ng/ml (fIR-PSA). Previous research suggests that inclusion by GS 7 may be associated with worse outcomes. Design setting and participants: We conducted a retrospective cohort study of US veterans diagnosed with fIR prostate cancer from 2001 to 2015. Outcome measurements and statistical analysis: We compared the incidence of metastatic disease, prostate cancer-specific mortality (PCSM), all-cause mortality (ACM), and receipt of definitive treatment between fIR-PSA and fIR-GS patients managed with AS. Outcomes were compared with those of a previously published cohort of patients with unfavorable intermediate-risk disease using cumulative incidence function and Gray's test for statistical significance. Results and limitations: The cohort included 663 men; 404 had fIR-GS (61%) and 249 fIR-PSA (39%). There was no evidence of difference in the incidence of metastatic disease (8.6% vs 5.8%, p = 0.77), receipt of definitive treatment (77.6% vs 81.5%, p = 0.43), PCSM (5.7% vs 2.5%, p = 0.274), and ACM (16.8% vs 19.1%, p = 0.14) between the fIR-PSA and fIR-GS groups at 10 yr. On multivariate regression, unfavorable intermediate-risk disease was associated with higher rates of metastatic disease, PCSM, and ACM. Limitations included varying surveillance protocols. Conclusions: There is no evidence of difference in oncological and survival outcomes between men with fIR-PSA and fIR-GS prostate cancer undergoing AS. Thus, presence of GS 7 disease alone should not exclude patients from consideration of AS. Shared decision-making should be utilized to optimize management for each patient. Patient summary: In this report, we compared the outcomes of men with favorable intermediate-risk prostate cancer in the Veterans Health Administration. We found no significant difference between survival and oncological outcomes.

Large scale crowdsourced radiotherapy segmentations across a variety of cancer anatomic sites.

Clinician generated segmentation of tumor and healthy tissue regions of interest (ROIs) on medical images is crucial for radiotherapy. However, interobserver segmentation variability has long been considered a significant detriment to the implementation of high-quality and consistent radiotherapy dose delivery. This has prompted the increasing development of automated segmentation approaches. However, extant segmentation datasets typically only provide segmentations generated by a limited number of annotators with varying, and often unspecified, levels of expertise. In this data descriptor, numerous clinician annotators manually generated segmentations for ROIs on computed tomography images across a variety of cancer sites (breast, sarcoma, head and neck, gynecologic, gastrointestinal; one patient per cancer site) for the Contouring Collaborative for Consensus in Radiation Oncology challenge. In total, over 200 annotators (experts and non-experts) contributed using a standardized annotation platform (ProKnow). Subsequently, we converted Digital Imaging and Communications in Medicine data into Neuroimaging Informatics Technology Initiative format with standardized nomenclature for ease of use. In addition, we generated consensus segmentations for experts and non-experts using the Simultaneous Truth and Performance Level Estimation method. These standardized, structured, and easily accessible data are a valuable resource for systematically studying variability in segmentation applications.

E pluribus unum: prospective acceptability benchmarking from the Contouring Collaborative for Consensus in Radiation Oncology crowdsourced initiative for multiobserver segmentation.

Purpose: Contouring Collaborative for Consensus in Radiation Oncology (C3RO) is a crowdsourced challenge engaging radiation oncologists across various expertise levels in segmentation. An obstacle to artificial intelligence (AI) development is the paucity of multiexpert datasets; consequently, we sought to characterize whether aggregate segmentations generated from multiple nonexperts could meet or exceed recognized expert agreement. Approach: Participants who contoured ≥ 1 region of interest (ROI) for the breast, sarcoma, head and neck (H&N), gynecologic (GYN), or gastrointestinal (GI) cases were identified as a nonexpert or recognized expert. Cohort-specific ROIs were combined into single simultaneous truth and performance level estimation (STAPLE) consensus segmentations. STAPLE nonexpert ROIs were evaluated against STAPLE expert contours using Dice similarity coefficient (DSC). The expert interobserver DSC ( IODSC expert ) was calculated as an acceptability threshold between STAPLE nonexpert and STAPLE expert . To determine the number of nonexperts required to match the IODSC expert for each ROI, a single consensus contour was generated using variable numbers of nonexperts and then compared to the IODSC expert . Results: For all cases, the DSC values for STAPLE nonexpert versus STAPLE expert were higher than comparator expert IODSC expert for most ROIs. The minimum number of nonexpert segmentations needed for a consensus ROI to achieve IODSC expert acceptability criteria ranged between 2 and 4 for breast, 3 and 5 for sarcoma, 3 and 5 for H&N, 3 and 5 for GYN, and 3 for GI. Conclusions: Multiple nonexpert-generated consensus ROIs met or exceeded expert-derived acceptability thresholds. Five nonexperts could potentially generate consensus segmentations for most ROIs with performance approximating experts, suggesting nonexpert segmentations as feasible cost-effective AI inputs.

Androgen deprivation therapy and acute kidney injury in patients with prostate cancer undergoing definitive radiotherapy.

BACKGROUND: Androgen deprivation therapy (ADT) is frequently utilized in conjunction with radiotherapy (RT) in the definitive management of prostate cancer. Prior studies have suggested an association between ADT use and acute kidney injury (AKI), however, these included heterogeneous populations undergoing a variety of treatments and relied on billing codes to ascertain the incidence of AKI. METHODS: We analyzed a cohort of 27,868 veterans undergoing definitive RT + /- ADT for prostate cancer between 2001 and 2015 using the Veterans Affairs Informatics and Computing Infrastructure (VINCI). Exposure was defined as use of ADT within one year of diagnosis. The primary outcome was AKI, defined by an increase in serum creatinine to at least 1.5 times the baseline value. AKIs were classified as mild, moderate, or severe in accordance with international guidelines. A multivariate competing risks model was used to account for demographic and oncologic factors as well as medications and procedures known to influence the risk of AKI. RESULTS: Most (n = 18,754) men received RT alone; 9,114 men received RT + ADT. The incidence of AKI at two years after diagnosis was 10.5% in the RT + ADT group and 7.9% in the RT group (Gray's test p < 0.01). Multivariate analysis confirmed ADT usage was associated with an increased risk for any AKI (SHR = 1.24, 95% CI = 1.14-1.36, p < 0.01). ADT was also associated with an increased risk of mild AKI (SHR = 1.13, 95% CI = 1.01-1.27, p = 0.04) and moderate AKI (SHR = 1.45, 95% CI = 1.20-1.76, p < 0.01), though not severe AKI (SHR = 1.33, 95% CI = 0.93-1.91, p = 0.11). CONCLUSIONS: Our findings confirm that use of ADT is associated with an increased risk of AKI in patients undergoing definitive RT for prostate cancer. Clinicians should be alert to the potential for renal dysfunction in this population.

Association Between Prostate-Specific Antigen Screening and Prostate Cancer Mortality Among Non-Hispanic Black and Non-Hispanic White US Veterans.

Importance: Black men have higher prostate cancer incidence and mortality than non-Hispanic White men. However, Black men have been underrepresented in clinical trials of prostate-specific antigen (PSA) screening; thus, there is a lack of data to guide screening recommendations for this population. Objective: To assess whether PSA screening is associated with reduced risk of prostate cancer-specific mortality (PCSM) among non-Hispanic Black men. Design, Setting, and Participants: This retrospective cohort study used data from the US Veterans Health Administration Informatics and Computing Infrastructure for men aged 55 to 69 years who self-identified as non-Hispanic Black or non-Hispanic White and were diagnosed with intermediate-, high-, or very high-risk prostate cancer from January 1, 2004, to December 31, 2017. Data were analyzed from August 2021 to March 2022. Exposures: Prostate-specific antigen screening rate, defined as the percentage of years in which PSA screening was conducted during the 5 years before diagnosis of prostate cancer. Main Outcomes and Measures: The primary outcome was risk of PCSM among Black men and White men. The association between PSA screening and risk of PCSM was assessed using Fine-Gray regression analysis. Risk of PCSM was also assessed categorically among patients classified as having no prior PSA screening, some screening (less than annual), or annual screening in the 5 years before diagnosis. Results: The study included 45 834 veterans (mean [SD] age, 62.7 [3.8] years), of whom 14 310 (31%) were non-Hispanic Black men and 31 524 (69%) were non-Hispanic White men. The PSA screening rate was associated with a lower risk of PCSM among Black men (subdistribution hazard ratio [sHR], 0.56; 95% CI, 0.41-0.76; P = .001) and White men (sHR, 0.58; 95% CI, 0.46-0.75; P = .001). On subset analysis, annual screening (vs some screening) was associated with a significant reduction in risk of PCSM among Black men (sHR, 0.65; 95% CI, 0.46-0.92; P = .02) but not among White men (sHR, 0.91; 95% CI, 0.74-1.11; P = .35). Conclusions and Relevance: In this cohort study, PSA screening was associated with reduced risk of PCSM among non-Hispanic Black men and non-Hispanic White men. Annual screening was associated with reduced risk of PCSM among Black men but not among White men, suggesting that annual screening may be particularly important for Black men. Further research is needed to identify appropriate populations and protocols to maximize the benefits of PSA screening.

Attitudes and access to resources and strategies to improve quality of radiotherapy among US radiation oncologists: A mixed methods study.

INTRODUCTION: We aimed to assess contouring-related practices among US radiation oncologists and explore how access to and use of resources and quality improvement strategies vary based on individual- and organization-level factors. METHODS: We conducted a mixed methods study with a sequential explanatory design. Surveys were emailed to a random 10% sample of practicing US radiation oncologists. Participating physicians were invited to a semi-structured interview. Kruskal-Wallis and Wilcoxon rank-sum tests and a multivariable regression model were used to evaluate associations. Interview data were coded using thematic content analysis. RESULTS: Survey overall response rate was 24%, and subsequent completion rate was 97%. Contouring-related questions arise in ≥50% of clinical cases among 73% of respondents. Resources accessed first include published atlases (75%) followed by consulting another radiation oncologist (60%). Generalists access consensus guidelines more often than disease-site specialists (P = 0.04), while eContour.org is more often used by generalists (OR 4.3, 95% CI 1.2-14.8) and younger physicians (OR 1.33 for each 5-year increase, 95% CI 1.08-1.67). Common physician-reported barriers to optimizing contour quality are time constraints (58%) and lack of access to disease-site specialists (21%). Forty percent (40%, n = 14) of physicians without access to disease-site specialists indicated it could facilitate the adoption of new treatments. Almost all (97%) respondents have formal peer review, but only 43% have contour-specific review, which is more common in academic centres (P = 0.02). CONCLUSION: Potential opportunities to improve radiation contour quality include improved access to disease-site specialists and contour-specific peer review. Physician time must be considered when designing new strategies.

Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review.

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physician-reported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning.

Immunotherapy and radiation therapy sequencing: State of the data on timing, efficacy, and safety.

Radiation therapy exerts a tumoricidal local effect as well as both local and systemic immunomodulation. Immune checkpoint blockade has become a widely used treatment modality across cancer types with a rapidly growing list of agents and US Food and Drug Administration-approved indications. Moreover, there may be synergy between radiation therapy and immune checkpoint blockade. Various strategies have been used, but the optimal sequencing of these therapies is unclear. In this review, the authors discuss the major mechanisms of available immune checkpoint inhibitors and explore the available preclinical and clinical evidence regarding treatment sequencing. They also review safety considerations and conclude with possible future directions.

A Systematic Review of Contouring Guidelines in Radiation Oncology: Analysis of Frequency, Methodology, and Delivery of Consensus Recommendations.

PURPOSE: Clinical trials have described variation in radiation therapy plan quality, of which contour delineation is a key component, and linked this to inferior patient outcomes. In response, consensus guidelines have been developed to standardize contour delineation. This investigation assesses trends in contouring guidelines and examines the methodologies used to generate and deliver recommendations. METHODS AND MATERIALS: We conducted a literature search for contouring guidelines published after 1995. Of 11,124 citations, 332 were identified for full-text review to determine inclusion. We abstracted articles for the intent of the consensus process, key elements of the methodology, and mode of information delivery. A Fisher exact test was used to identify elements that differed among the guidelines generated for clinical trials and routine care. RESULTS: Overall, 142 guidelines were included, of which 16 (11%) were developed for a clinical trial. There was an increase in guideline publication over time (0 from 1995-1999 vs 65 from 2015- 2019; P = .03), particularly among recommendations for stereotactic radiation and brachytherapy. The most common disease sites were head and neck (24%), gastrointestinal (12%), and gynecologic (12%). Methods used to develop recommendations included literature review (50%) and image-based methods (45%). Panels included a median of 10 physicians (interquartile range, 7-16); 70% of panels represented multidisciplinary expertise. Guidelines developed for a clinical trial were more likely to include an image-based approach, with quantitative analysis of contours submitted by the panel members and to publish a full set of image-based recommendations (P < .005). CONCLUSIONS: This review highlights an increase in consensus contouring recommendations over time. Guidelines focus on disease sites, such as head and neck, with evidence supporting a correlation between treatment planning and patient outcomes, although variation exists in the approach to the consensus process. Elements that may improve guideline acceptance (ie, image-based consensus contour analysis) and usability (ie, inclusion of a full image set) are more common in guidelines developed for clinical trials.

Development and Usage of eContour, a Novel, Three-Dimensional, Image-Based Web Site to Facilitate Access to Contouring Guidelines at the Point of Care.

PURPOSE: Variation in contouring quality by radiation oncologists is common and can have significant clinical consequences. Image-based guidelines can improve contour accuracy but are underused. We sought to develop a free, online, easily accessible contouring resource that allows users to scroll through cases with 3-dimensional images and access relevant evidence-based contouring information. MATERIALS AND METHODS: eContour (http://econtour.org) was developed using modern Web technologies, primarily HTML5, Python, and JavaScript, to display JPEGs generated from DICOM files from real patient cases. The viewer has standard tools for image manipulation as well as toggling of contours and overlayed images and radiation dose distributions. Brief written content references published guidelines for contour delineation. Mixpanel software was used to collect Web page usage statistics. RESULTS: In the first 3 years of operation (March 8, 2016 to March 7, 2019), a total of 13,391 users from 128 countries registered on the Web site, including 2,358 physicians from the United States. High-frequency users were more likely to be physicians (P < .001) and from the United States (P < .001). In one 6-month period, there were 68,642 individual case page views, with head-and-neck the most commonly viewed disease site (32%). Users who accessed a head-and-neck case were more likely to be high-frequency users, and 67% of repeat users accessed the same case more than once. CONCLUSION: The large, diverse user base and steady growth in Web site traffic over the first 3 years of eContour demonstrate its strong potential to address the unmet need for dissemination and use of evidence-based contouring information at the point of care.

The impact of daily bladder filling on small bowel dose for intensity modulated radiation therapy for cervical cancer.

Research demonstrates that instructing patients to have a full bladder for pelvic radiotherapy results in highly variable bladder volumes at daily treatment. We aimed to determine bladder volume variation in patients with intact cervical cancer treated with intensity-modulated radiotherapy (IMRT) on an empty bladder and estimate the difference in radiation dose to the small bowel compared to treating on a full bladder. We identified 29 patients treated with IMRT from 2010 to 2013 who underwent 2 planning computed tomography (CT) scans, 1 with a full bladder followed by 1 with an empty bladder. Interfractional variation in bladder volume was measured using 782 daily cone beam computed tomography (CBCT) scans. To estimate dose to small bowel, radiation plans were created on both empty and full bladder CT scans using an automated knowledge-based planning modeling program. Mean bladder volume with empty bladder instructions was 67 ± 26 cc compared to 91 ± 43 cc for no bladder instructions and 154 ± 54 cc for full bladder instructions (p < 0.001). There was a significant reduction in the absolute bladder volume variation in patients given empty bladder instructions compared to full bladder instructions (p < 0.05) The intraclass correlation coefficient showed low reliability of bladder filling across all groups (p = 0.6). The average bowel V45 for the empty bladder plans was 188 cc, compared to 139 cc for the full bladder plans (p < 0.05). More plans created on an empty bladder exceeded Quantitative Analyses of Normal Tissue Effects in the Clinic (QUANTEC) guidelines but this was not significant (31% vs 14%, p = 0.12). Reliability of bladder volume at the time of radiation treatment is low, regardless of bladder filling instructions, although an empty bladder reduces absolute variation in bladder volume. Radiation planning on an empty bladder predicts a larger volume of small bowel receiving 45 Gy compared to a full bladder, although bowel dose on average is still within QUANTEC guidelines (V45 < 195 cc).

An interactive contouring module improves engagement and interest in radiation oncology among preclinical medical students: Results of a randomized trial.

PURPOSE: Studies have shown significant gaps in knowledge of radiation therapy among medical students and primary care providers. The goal of this study was to evaluate the effect of an interactive contouring module on knowledge and interest in radiation oncology among preclinical medical students. METHODS AND MATERIALS: Second-year medical students at the University of California, San Diego were randomized to participate in an interactive contouring exercise or watch a traditional didactic lecture on radiation oncology. Participants completed knowledge tests and surveys at baseline, immediately following the exercise, and 3 months later. Statistical analysis included Wilcoxon signed-rank test for pre- and posttest comparisons and Wilcoxon rank sum test for comparison between groups. RESULTS: Forty-three medical students participated in the trial (21 in the didactic group; 22 in the contouring group). Students completing the contouring module demonstrated similar overall knowledge improvement compared with the traditional didactic group (+8.6% vs +6.6%, not significant) but endorsed greater engagement on a 5-point Likert-type scale (3.10 vs 3.76, P = .02). At 3-month follow-up, there was a nonsignificant trend toward improved overall knowledge in the contouring group (43% vs 51%, P = .10), with a significance difference in a subset of questions on knowledge of the process of radiation therapy as well as side effects (51% vs 75%, P = .002). Students in the contouring group demonstrated more interest in pursuing a clinical radiation oncology rotation (2.52 vs 3.27, P = .01). CONCLUSIONS: Use of an interactive contouring module was an effective method to teach preclinical medical students about radiation oncology, with no significant difference in knowledge gained compared with a traditional didactic lecture; however, higher engagement among students completing the contouring module led to improved retention of knowledge of radiation side effects and greater interest in radiation oncology. These data suggest a potential benefit of integrating an interactive radiation oncology module into the preclinical medical school curriculum.